GRADE guidelines 33: Addressing imprecision in a network meta-analysis.

OBJECTIVE: This article describes GRADE guidance for assessing imprecision when rating the certainty of the evidence from network meta-analysis. STUDY DESIGN AND SETTING: A project group within the GRADE working group conducted iterative discussions, computer simulations, and presentations at GRADE working group meetings to produce and obtain approval for this guidance. RESULTS: When addressing imprecision of a network estimate, reviewers should consider the 95% confidence or credible interval, and the optimal information size. If the 95% confidence or credible interval crosses a pre-specified threshold, reviewers should rate down the certainty of the evidence. If the 95% confidence interval does not cross any pre-specfied threshold, reviewers should consider the optimal information size. Because addressing the optimal information size may be challenging, reviewers can use the effect size to decide if any calculations are necessary. When the size of the effect is modest or the optimal information size is met, reviewers should not rate down for imprecision. CONCLUSION: Reviewers should use this guidance when to appropriately address imprecision in the context of the assessment of certainty of evidence from network meta-analysis.

Guidelines that use the GRADE approach often fail to provide complete economic information for recommendations: A systematic survey.

OBJECTIVE: Little is known about how developers and panel members report cost and cost effectiveness considerations in GRADE guideline Evidence-to-Decision (EtD) frameworks. A systematic survey was conducted to explore approaches and factors contributing to variability in economic information reporting. STUDY DESIGN AND SETTING: Guideline organization websites were systematically searched to create a convenience sample of guidelines. Reviewers screened published EtD frameworks and generated frequencies of reporting approaches. We used thematic analysis to summarize factors related to variability of economic information reporting. RESULTS: We included 142 guidelines. The overall rate of reporting economic information was high (91%); however, there was variability across completion of predefined EtD Likert-type judgments (70%), noting information as not identified across EtD framework domains (57%), and providing remarks to justify recommendations (38%). Six themes contributing to variability emerged, related to: intervention, population, payor, provider, healthcare resource use, and economic model building factors. Only 2 guidelines performed a GRADE certainty appraisal of economic outcomes. CONCLUSION: Completing predefined EtD Likert-type judgments, specifically reporting a literature review approach, study selection criteria and economic model building limitations, as well as linking these to recommendation justification remarks are potential areas for improved use, adoption and adaptation of recommendation, and transparency of GRADE EtD frameworks.

Use of GRADE in evidence syntheses published in high-impact-factor nutrition journals: A methodological survey.

OBJECTIVE: To identify and describe the use of the GRADE approach for rating the certainty of evidence in nutrition systematic reviews (SRs). STUDY DESIGN AND SETTING: We systematically searched for SRs using GRADE that were published between 2015 and 2019 in the 10 "nutrition" journals with the highest impact factor according to the JCR 2018. RESULTS: Out of 800 SRs, 55 SRs of randomized control trials (RCTs) and/or nonrandomized studies (NRSs) used GRADE. Forty-seven SRs (5.9%) rated the outcome specific certainty of evidence (n = 36 in 2018/2019). We identified a total of 465 certainty of evidence outcome ratings (n = 335 RCT ratings), ranging from very-low (28.8%) to low (41%), moderate (26.5%), and high (3.7%). Very-low and high certainty of evidence ratings accounted for 61.4% and 0.8% of ratings in SRs of NRSs, compared to 16.1% and 4.8% in SRs of RCTs. Certainty of evidence was downgraded mostly for risk of bias (37.8%) and imprecision (33%) in SRs of RCTs and for imprecision (32.7%), risk of bias (29.4%) and inconsistency (29%) in SRs of NRSs. CONCLUSION: Our study suggests a need for directing more attention toward strengthening acceptance of GRADE as well as building knowledge of the GRADE methodology in nutrition evidence synthesis.

Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies.

BACKGROUND: Nutrition and physical activity policies have the potential to influence lifestyle patterns and reduce the burden of non-communicable diseases. In the world of health-related guidelines, GRADE (Grading of Recommendations, Assessment, Development and Evaluation) is the most widely used approach for assessing the certainty of evidence and determining the strength of recommendations. Thus, it is relevant to explore its usefulness also in the process of nutrition and physical activity policymaking and evaluation. The purpose of this scoping review was (i) to generate an exemplary overview of documents using the GRADE approach in the process of nutrition and physical activity policymaking and evaluation, (ii) to find out how the GRADE approach has been applied, and (iii) to explore which facilitators of and barriers to the use of GRADE have been described on the basis of the identified documents. The overarching aim of this work is to work towards improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior. METHODS: A scoping review was conducted according to current reporting standards. MEDLINE via Ovid, the Cochrane Library, and Web of Science were systematically searched up until 4 July 2019. Documents describing a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as "guidelines," or systematic reviews used to inform policymaking were included. RESULTS: Thirty-six documents were included. Overall, 313 GRADE certainty of evidence ratings were identified in systematic reviews and guidelines; the strength of recommendations/policies was assessed in four documents, and six documents mentioned facilitators or barriers for the use of GRADE. The major reported barrier was the initial low starting level of a body of evidence from non-randomized studies when assessing the certainty of evidence. CONCLUSION: This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions (policymaking), as well as in the development of guidelines intended to guide policymaking. Several areas for future research were identified to explore the use of GRADE in health policymaking and evaluation.